This invention relates to a method and apparatus for processing image data, and in particular for processing a sequence of images of a non-rigid body to correct for movement of the non-rigid body between the images.
There are many applications in which it is useful to detect corresponding features in successive images of a body, for instance to allow detection of the movement of the body and to correct for movement of it. Many methods have been proposed for detecting such corresponding features. The successful detection of corresponding features in successive images allows the registration of the images, i.e. allows subsequent images to be transformed to eliminate differences caused by motion.
A common way to detect motion in successive images is to use landmark and segmentation based algorithms (also known as sparse matching). In such methods, geometric features which can be recognised in each image are matched, and these features are used to generate a transformation which can be applied to the whole image. However, this method is not useful for images which lack recognisable geometric features.
For instance, in the field of diagnosis of breast cancer, an imaging technique which is currently being explored is magnetic resonance imaging (MRI). FIG. 1 of the accompanying drawings shows schematically the arrangement for MRI of a human breast in which the patient 1 lies prone inside the main MR coil 3 on a couch 5. With this arrangement the breasts hang pendulously inside the double breast coil 7 as shown diagrammatically in FIG. 2. This technique has certain advantages as compared with, for instance, x-ray mammography, in that its use is not restricted to post-menopausal women or to breasts which do not include scar tissue, and it is also a 3-D technique (actually producing a set of vertically displaced 2-D slices). However, it does have certain problems. Firstly, abnormalities in the breast cannot be distinguished in a standard MRI image. To detect abnormalities it is necessary to inject a paramagnetic compound (typically Gd-DTPA) as a contrast agent and then scan every minute for approximately 7 minutes to monitor the take up of the contrast agent. Each scan consists of 16 slice images in, for instance, the X, Y plane as shown in FIG. 1. Because of the length of time over which imaging occurs, it is very likely that the patient will move during that time. These movements make it difficult, or sometimes impossible, to interpret the succession of images because not only are the images changing through take up of the contrast agent, but also because of movement of the patient. For instance FIG. 3(A) illustrates a pre-contrast image and FIG. 3(B) a post-contrast image in which the patient relaxed the pectoral muscle between the two images. The pectoral muscle can be seen as the darker grey tissue towards the top of the image. It can be seen that the effect of relaxation is to change the shape of the breast and cause non-rigid movement in the tissue surrounding the muscle. Such non-rigid movement is difficult to compensate for.
It will also be appreciated from FIG. 3 that the images are not of the type in which there are clearly recognisable geometric features which can help to detect the movement between images. Further, because the contrast agent is being taken up during the imaging process, features which have one level of brightness in a first image will not necessarily have the same value of brightness in a subsequent one (the image is said to be non-conservative). In fact, the features which are of most interest (abnormalities such as tumours) take up the contrast agent quicker than other tissue and so inevitably, and deliberately, will be changing in intensity between the different images.
Thus these images are of the type which require correction for motion, but which are not susceptible to normal motion detection techniques.
The present invention provides a technique for detecting and correcting successive images for movement where the movement is non-rigid-body movement and where the images may be themselves changing as a function of time. It is therefore applicable to images which have non-conservative flow, i.e. the total amount of brightness in the image changes. The invention is not only applicable to MRI techniques, but other non-conservative image flows where there is movement between images.
The present invention achieves this by calculating and storing for each of a plurality of sampling points in the image a probability map for the displacement of that point between the image and a time separated image. Thus not only one possible displacement is considered for each sampling point, but instead all are considered. This probability map, namely the probabilities that the sampling point has moved in a particular direction, is then refined in an iterative process by considering the probability maps for sampling points neighbouring it.
Thus whereas with previously proposed techniques only one “optimal” displacement vector was considered for each sampling point, with the present invention all are considered and stored for future processing. With the previous techniques a first estimate of the motion field was made based on the single optimal vector for each point, and then the field was refined by iteratively recomputing those vectors. However, in non-rigid flow there can be several plausible motions locally and so algorithms which ignore that fact can fall into local minima which do not correspond to the ultimately optimal solution during the iterative process. Because the present invention retains the fact that there are several possible alternatives to the “optimal” displacement vector, it can avoid being trapped in incorrect local minima.
Thus in more detail, the present invention provides a method of processing image data of a plurality of time-separated images of a non-rigid body to detect movement of the body, comprising the steps of:—
for each of a plurality of sampling points in each image calculating and storing a plurality of candidate movements together with the estimated probability of each candidate;
iteratively recalculating for each sampling point the probability of each of the candidate movement based on the stored probability of that candidate movement and the probabilities of the candidate movements at other sampling points; and
generating from the recalculated probabilities a motion field indicative of the non-rigid body movement.
The sampling points are set as a regular array over the image and the said other sampling points which are considered in the recalculation process are preferably neighbouring sampling points. The sampling points need not correspond to individual pixels in the image, but are preferably spaced by several pixels.
The estimated probabilities for the candidate movements are found by using a similarity measure which indicates the similarity of each sampling point to sampling points in the preceding image. The similarity measures can be converted into probabilities by simply normalising them for each sampling point so that they sum to unity. The candidate movements are, of course, the vector displacements which map the respective sampling points in the preceding image to the current sampling point.
As mentioned above a problem with non-conservative images such as MRI images when a contrast agent is being dynamically taken-up, is that points of a certain intensity in one image cannot be expected to have the same intensity in another. The present invention overcomes this problem by careful choice of similarity measure, and in particular by selecting the similarity measure from mutual information, normalised mutual information or entropy correlation coefficient.
The iterative recalculation of the stored probabilities comprises multiplying each stored probability by the product of the stored probabilities for the neighbouring sampling points. Preferably the only maximum probability for each of the neighbouring points is taken, more preferably weighted according to the difference between the candidate movement and the maximum probability movement of the neighbouring sampling point. The recalculation can also be limited to use only movements which are judged to be similar to the movement of the sampling point under consideration, for instance where the magnitude of the displacement caused by the movements differ by less than a preset amount.
The number of iterations in the recalculation can be set as desired, conveniently according to the distance between salient points in the image.
Once the probabilities of the candidate movements have been refined by the iterative process, the motion field is generated by selecting as the movement at each sampling point that candidate movement which has the maximum probability. This motion field can then be used to correct later images in the sequence by converting the motion field into a transformation which can be applied to each pixel. The motion field can be converted to a transformation either by interpolation or by fitting a parametric transformation to the motion field.
Having corrected the images it is possible to repeat the process using differently spaced sampling points, for instance more closely spaced sampling points, and preferably to do this iteratively to further refine the detection of the motion field and registration of the different images.
The invention also provides a corresponding apparatus for processing image data and the invention can be embodied as a computer program which may, for instance, be stored on a computer readable storage medium.